Searching PAJ http://vvwwl9.ipdl.ncipi.go jp/PAl/result/deteil/imin/wAAAv^ 



PATENT ABSTRACTS OF JAPAN 

(ll)Publication number : 09-026850 
(43)Date of publication of application : 28.01.1997 



(51) Int. CI. G06F 3/033 

G06F 3/03 



(21) Application number : 


07-173842 


(71) Applicant : 


CANON INC 


(22) Date of filing : 


11. 07. 1995 


(72) Inventor : 


OZAKA TSUTOMU 








NAGANUMA TADASHI 








YOMO MAKOTO 








ISONUMA TOMOYUKI 








KAWAMURA KOJI 



-lio' 




fll! 



V 

■If 

F 



114 



(54) USER INTERFACE DEVICE 

(57) Abstract: 

PROBLEM TO BE SOLVED: To remarkably improve the 
operability of a user interface device by feeding 
the information on the operation of a pointing 
device back to a user and processing the 
information on the user' s operation of the 
pointing device in response to the feedback of the 
information on the pointing device operation. 
SOLUTION: When a pointer which is operated by a 
mouse 114 serving as a pointing device approaches 
to or passes through an object or its periphery, 
the pointer is properly operated by a reaction 
control part 1 of the mouse 114 and under the 
control of a host computer 110. Then the operation 
of the mouse 114 and the information given from 
the mouse 114 are fed back to a user. The 
information on the mechanical actions like 
vibrations, impacts, etc, , that is, the non-visual 
information on the user' s operation of the mouse 

114 are processed in response to the preceding feedback of information. In such a 
constitution, the reactions and the touch of the mouse 114 are controlled and the 
operability is remarkably improved for a user interface device. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect 
the original precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to user interface equipments, such 
as user interface equipment especially a mouse, a tablet, and a trackball. 
[0002] 

[Description of the Prior Art] In order to operate a computer or to input data 
conventionally, input units, such as a keyboard, a tablet, a mouse, and a trackball, 
are used. Especially the tablet and mouse that can input coordinate information and 
the information on a locus into a computer while looking at a monitor are used with 
a graphical user interface in many cases, and can raise the operability of a 
computer. 

[0003] The so-called relation with a graphical user interface method in recent years 
of especially a mouse is deep, by clicking and dragging an icon and a window, the 
intuitive actuation of it is attained and it is widely received also in the general 
user. 

[0004] Drawing 12 is the external view of the conventional mouse, and the mouse 
button to which the housing of a mouse operates a sign 100 and a user operates 101 
and 102, and 103 are the lead wire for receiving a host computer, data I/O, and 
current supply. 

[0005] Drawing 13 shows the side face of drawing 12 , and drawing 14 shows the base 
of drawing 12 , and the sign 104 shows the detection ball which carried out the 
shape of a globular form for detecting the location of a mouse. Like illustration, 
the detection ball 104 is projected from opening 104a of the base of a mouse, and it 
is supported by the support device in which it does not illustrate so that it can 
rotate freely according to the direction and amount of migration of a mouse. Signs 
105 and 106 are the slide members which stabilize slipping of a mouse, and consist 
of resin members, such as polyacetal and Teflon. 

[0006] Drawing 15 shows the whole system which uses a mouse 114. A sign 110 is a 
host computer, it is mounted with OS (operation system) 111, OS, or an application 
level, and including the mouse interface 113 for performing the mouse driver 112 for 
performing input/output control to a mouseT^nd hardware connection with a mouse 
etc., it is constituted here so that various application software can be performed. 
[0007] In drawing 15 , a sign 114 is a mouse and has the carbon button detection 
section 116 which detects the ON/OFF condition of the movement magnitude detecting 
element of 115, or mouse buttons 101 and 102. 
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[0008] Drawing 16 shows the typical method of the movement magnitude detection 
section of a mouse. Like illustration, the detection rollers 120 and 121 which 
rotate to 1 shaft orientations and possess the slit of the shape of an optical disk 
touch the detection ball 104 in the interior of a mouse. On the detection rollers 
120 and 121, the component in each direction of two direct axes of coordinates is 
detected by the detection sensors 122 and 123 among migration [ in / for rotation of 
the encoder plates 120a and 121a / the installation flat surface of a mouse ]. A 
sign 124 is a pressure roller for making a detection roller produce frictional 
force. 

[0009] Here, although the configuration which detects movement magnitude for a 
mechanical motion of a ball through an optical encoder was shown, various methods 
using the method which reads the pattern of a mouse installation flat surface 
directly optically as a detection means, and a magnetic change, such as a thing, are 
proposed. 
[0010] 

[Problem (s) to be Solved by the Invention] The above conventional mice are the means 
of a user interface, and they are used in order to tell a user' s volition to a host 
computer. 

[0011] That is, the conventional mouse has an indispensable display etc., in order 
to feed back to a user the information which performs an information input to a 
computer and was inputted into it by one-way traffic from a user, and it has the 
following problems. 

[0012] 1) While operating the object on monitors, such as an icon and a drag 
(actuation which is made to move a mouse where a mouse button is pressed, and pulls 
objects, such as an icon and a window, on a display) of a window, there is no check 
means except seeing a display. 

[0013] 2) In case an icon and a window are chosen, it is necessary to place a 
pointer on an object correctly. 

[0014] Such a problem is the same even when it is going to constitute the same 
graphical user interface as the above from pointing devices, such as a mouse, a 
trackball, and a tablet. 

[0015] The technical problem of this invention is to offer the user interface 
equipment which can improve operability greatly, when the above problem is solved 
and it constitutes a graphical user interface using pointing devices, such as a 
mouse, a trackball, and a tablet. 
[0016] 

[Means for Solving the Problem] In order to solve the above technical problem, it 
sets to this invention. A feedback means to feed back information to a pointing 
device by mechanical works, such as reaction force, vibration, and an impact, to 
actuation of a user' s pointing device at a user, In the operating environment using 
the graphical user interface whi ch arranges ob jects, such as an icon, a window, and 
a pop up menu, and chang es on a display In case the pointer operated by tne pointing 
{^device passes and approaches tfie object for actuation, or its n ear, said fe edback 
means is minded for the information about actuation of a potrvfing device. While 
feeding back to the user, the configuration which established the control means 
which processes the information about actuation 'of said pointing device of the user 
according to this feedback was adopted. 
[0017] ■ 



2 of 10 



7/31/06 9:45 AM 



JP,09-026850A [DETAILED DESCRIPTION] 



http://www4.ipdl.ncipi.go.jp/cgi-bin/tran_web__cgi_ejje 



[Function] Since according to the above configuration the control means which 
processes the information about actuation of said pointing device of the user 
according to this feedback is established while feeding back the information about 
actuation of a pointing device to a user through said feedback means, the user 
interface through means other than a visible display becomes possible. 



[Example] Hereafter, this invention is explained to a detail based on the example 
shown in a drawing. 

[0019] In this invention, in case a user operates a pointing device, by generating 
mechanical works, such as vibration, reaction force, and an impact, that response 
and feel are controlled and the information inputted from actuation of a pointing 
device thru/or a pointing device through this response and feel is fed back to a 
user. In addition, below, a mouse is mainly shown as an example. 
[0020] <lst example) drawing 1 shows the structure of the computer system in which 
such actuation is possible, and differing from the conventional example of drawing 
12 is the point that the reaction force control section 1 is formed in the mouse 



[0021] Drawing 2 shows the structure of a mouse 114. Here, the reaction force 
control section 1 which consisted of each part material of signs 10-14 is formed in 
a mouse like illustration. 

[0022] the electromagnet ism which generates a magnetic field in the current value 
arranged in a mouse that the magnet 13 fixed at arm 11 and arm 11 tip supported by 
the center of rotation 12 free [ rotation ] that the friction pad 10 and friction 
pad 10 which have suitable coefficient of friction for this reaction force control 
section 1 sliding with the detection ball 104, and braking this should be supported, 
and a magnet 13 should be opposed — it consists of a coil 14. 
[0023] the reaction force control section 1 — electromagnet ism — a pressure is 
produced with the suction force of a coil 14 and a magnet 13 between the detection 
balls 104 for detecting a friction pad 10 and migration of a mouse for the center of 
rotation 12 at the supporting point. The frictional force then produced generates 
reaction force. 

[0024] the strength of the reaction force at this time — electromagnet ism — it is 
controllable by the current value passed in a coil, moreover — in order to cancel 
control of the reaction force control section 1, or it energizes beforehand by means 
of a spring etc. — electromagnet ism — the direction which a coil 14 and a magnet 
13 repel — electromagnetism — the current of a coil 14 is passed. 
[0025] as a means to control reaction force at this time — electromagnetism — 
which electric machine sensing elements, such as a piezoelectric device and a 
piezo-electric element, may be used, without restricting to the approach using a 
coil and a magnet. 

[0026] The reaction force control section 1 is suitably operated by control from a 
host computer 110, and it can be used in order to feed back the information from 
actuation of a mouse or a mouse to a user. For this reason, the signal line for 
controlling a reaction force control section from a host computer is added to the 
mouse interface. 

[0027] Drawing 3 is the example of the graphical user interface which is operating 
by this example. Drawing 3 is a drawing showing the screen of the monitor TV 
connected to the host computer, and the icon whose sign 30 is an object for a menu 



[0018] 
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bar and 33 to display a disk file and an I/O device on a window, and for the title 
bar of a window and 32 display virtually 34a, 34b, 34c, 34d, and 34e on a user in 
the outer frame of the boundary of the outermost part of a window and 31, and 35 are 
the pointers of a mouse, 

[0028] Thus, with the interface using objects, such as an icon, a window, and a 
menu, objects, such as a directory on a file system, and an icon related with 
application software etc., a window, a menu, may be moved for the purpose, such as 
vacating the location for doing such an activity on migration of a file, copy, 
deletion, I/O with a peripheral device, and a display on the window of a desktop. 
[0029] The mouse of this example can be used for the following applications in such 
migration actuation. [0030] 1) When a pointer passes through an icon and window top, 
generate reaction force. For example, v^in drawing 3 , a user moves the location of a 
pointer 35 to a menu bar 32, a window 33, and Icons 34a'-34e with a mouse, and 
migration etc. is performed by choosing and operating it by mouse buttons 101 or 
102. 

[0031] In order to choose with a pointer 35 at this time, for example, icon 34b in 
drawing, when moving a pointer 35, if a pointer is on an icon, the reaction force 
control section 1 will be operated, and it is made to recognize that a pointer is in 
the location which can choose an icon as a user by making the feeling of use of the 
mouse at the time of actuation heavier than usual. 

[0032] Not only an icon but also when dragging other objects 33, for example, a 
window etc. , same reaction force control can be performed. 

[0033] 2) When recognizing the display of a screen or passing near the small object 
which is difficult to choose, generating reaction force is also considered. For 
example, in the application of an image-processing system, a perpendicular 62 may be 
drawn on a line 61 from a point 60 like drawing 4 by a user operating a mouse in a 
window 33, for example, moving a pointer 35. 

[0034] If the reaction force control section 1 is operated and a mouse button 101 or 
102 is pushed in the meantime when it is in the predetermined distance near this 
point 60, since it is hard to recognize this point 60 very small to other objects in 
case the point 60 of the inside at this time, for example, drawing, is chosen and a 
line is drawn, it will be regarded as what has recognized this point 60. Reaction 
force control can be similarly performed to the object which it is small and is hard 
to recognize not only by the point but by a line etc. as it is in this example. 
[0035] Thus, according to reaction force generating of a mouse, it feeds back, and 
the display of a screen can be recognized or actuation of a small object which is 
difficult to choose can be made easy. 

[0036] It can b e determined by the mag nitude of an object whether feedback by such 
reaction torce generating is perf ormed. For example, level or when the vertical 
number of dots is below constant value7 the^^^ 

object and it and which is been made to perform reaction force generating to the 
object can be considered. Thereby, reaction for'cie~l5oritTol can be perfofiiied now only 
to an obje ct .smaller than r ^^rtain magn Uude and size. 

[0037] 3) The actuation condition of a menu can also be fed back by generating 
reaction force at the time of menu reversal. 

[0038] For example, in a graphical user interface as shown in drawing 5 , menu 
selection may be performed by dragging the entry of a menu bar 32. In drawing 5 , 
the item in a pull down menu like illustration is chosen by dragging a certain menu 
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(here "edit'') with the pointer 35 operated with a mouse (or click). Into this drag, 
the sequential reversal location moves [ each entry ] as the entry of cut on a menu, 
copy, paste, and clear is passed, and an item is chosen by releasing the depression 
of a mouse button in a desired location etc. 

[0039] Moreover, although the entry which cannot be chosen in a pull down menu may 

usually exist in such an environment by the mode of operation of application to 

which the I/O device is not connected, the user is made to recognize with means, 

such as indicating dark the entry in which such selection is impossible. 

[0040] Thus, when operating the pull down menu with which a selectable thing and an 

impossible thing are intermingled, and an entry is selectable, it is possible to 

apply reaction force by the reaction force control section 1 of a mouse. 

[0041] If it does in this way, a user can be made to recognize the entry in which 

the selection in a pull down menu is certainly more nearly impossible than feedback 

of only the conventional display. 

[0042] Thus, with a mouse with a reaction force control section as shown in drawing 
1 and drawing 2 , when a pointer is in an object or near in the graphical user 
interface environment where objects, such as an icon and a window, are operated, or 
when it is in the operational location of PURUDAUN, information other than vision 
can be fed back to a user by making a mouse produce reaction force. 
[0043] Namely, it is based on mouse positional information (or pointer operated by 
that cause). When a pointer is in an object or near, or when it is in the 
operational location of PURUDAUN, by feeding back actuation and input by reaction 
force control The input from actuation of a mouse or a mouse can be fed back to a 
user, the actuation of a mouse and the feedback of the information from a mouse 
which were conventionally performed through the display can be assisted, and a 
user' s burden can be mitigated. 

[0044] Moreover, it may be made to perform reaction force generating to a conversely 
selectable object rather than performs reaction force generating to the object which 
cannot be operated and chosen (refer to the below-mentioned example). For example, 
in the case of the above-mentioned pull down menu, in case a selectable entry is 
passed, it may be made to perform reaction force generating. 

[0045] What is necessary is just to set the pointer and object on which reaction 
force control is made to act as arbitration on the level of OS or application 
software. 

[0046] Although the example which feeds back actuation and input by reaction force 
control based on mouse positional information (or pointer operated by that cause) 
was shown above <the 2nd example), it is also further considered to be a mouse to 
add the control according to mouse passing speed (or pointer operated by that 
cause) . 

[0047] Drawing 6 shows the structure of the computer system of the 2nd example, and 
a different point from the 1st example of drawing 1 is a point that the mouse 
passing speed detection section 2 is formed in the mouse driver 112. 
[0048] This mouse passing speed detection section 2 can detect the passing speed of 
a mouse the positional information from a mouse, and by usually carrying out 
counting of the pulse signal within predetermined time amount, and it enables it to 
always use this signal within the host computer. 

[0049] In the 1st example, since a user operates a mouse, moves a pointer, reaction 
force is made to generate according to positional information during the migration 
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and reaction force is always generated near [ where reaction force control is set up 
] all the objects, a user's feeling of use may be reduced remarkably. The 2nd 
example solves this trouble. 

[0050] It is characteristic in the 2nd example to pay one' s attention to the 
actuation in case a user operates a mouse. 

[0051] Usually, when choosing icon 34b using a pointer 35 on a graphical user 
interface like drawing 3 , a user moves a pointer early, when a pointer is far from 
an icon, and if it approaches, making the rate late is known. 

[0052] In the 2nd example, the object judges whether it is the object of a user' s 
purpose from the property that a user changes the passing speed of a pointer in this 
way. 

[0053] Drawing 7 is the flow chart having shown the situation of the mouse 
input/output control in the host computer 110 of **** 2 example. This control is 
performed by CPU of a host computer 110. 

[0054] If the passing speed of a mouse is a rate decision means to compare and judge 
the value acquired [ whether it is below a predetermined value and ] from the mouse 
passing speed detection section 2, and is below a predetermined value and step S71 
is truth (true) and beyond a predetermined value, it will be judged to be a false 
(false). 

[0055] Step S72 is a mouse location decision means to judge the relation between the 
location of a pointer, and the location of objects, such as an icon and a window, 
and if the location of a pointer is the inside of an object, or near, it will be 
judged to be truth and a certain predetermined location detached building 

^^^^^^^^^ 

[0056] Step S73 is a reaction force control means which makes the reaction force 
control section 1 of a mouse 114 actually generate reaction force. 
[0057] In the 2nd example, the result of the mouse scaling decision means 71 is 
below a predetermined rate, and when the location of a pointer is located in the 
inside of an object, or near, a reaction force control means operates and generates 
reaction force by the reaction force control section 1, so that clearly from drawing 
7 . 

[0058] That is, even if the location of a pointer is the inside of an object, or 
near, if the passing speed at that time is over the predetermined rate, a user will 
judge that it has not aimed at actuation of that object, and reaction force control 
will not be performed in this case. 

[0059] Although a user's feeling of use is spoiled in the 1st example since reaction 
force is always generated near [ where reaction force control is set up since 
reaction force control is performed only based on positional information ] all the 
objects, such a problem is avoidable in this example. 

[0060] In addition, the procedure of processing of the mouse scaling decision means 
of drawing 7 and a mouse location decision means may be reverse. 

[0061] When the carbon button of a mouse is further operated by the 2nd example near 
the object, <3rd example> this example passes through an object (near) location, and 
when reaction force is generated, it adds the processing which amends the positional 
information which actuation produced using the hour entry of a from. 
[0062] When a mouse button is specifically pressed after beginning reaction force 
control in the 2nd example, after beginning reaction force control [ near the object 
], the time amount of the time of a mouse button being pressed is measured, and if 
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it is in predetermined time amount, a user will interpret it as a thing with the 
volition of selection of the object, will transpose the location of a pointer to the 
location which began reaction force control, and will process a host computer. 
[0063] Although drawing 8 is the flow chart having shown the description of this 
example 3 and steps S71-S73 are the same as the 2nd example, if the location of a 
pointer is the inside of an object, or near at step S72, in step S74, the location 
(namely, location which begins reaction force control at step S73) will be 
memorized. 

[0064] That is, step S74 is a coordinate position-memory means to memorize 
temporarily the location of a pointer when reaction force control is started, and is 
memorized by the memory usually mounted in the host computer 110. 
[0065] if step S75 is a carbon button decision means to judge whether the mouse 
button was pressed and a carbon button is pushed — truth — if not pushed, it 
judges with a false. 

[0066] It is a time amount decision means, step S76 compares time amount until step 
S75 as a carbon button decision means consists of reaction force control truly with 
the predetermined time amount set up beforehand, and if it becomes below a 
predetermined value and it will be truth and beyond a predetermined value, it will 
judge it to be a false. 

[0067] Step S77 is a coordinate processing means which transposes the coordinate 
location of a use current mouse to the coordinate value memorized at step S74 (it 
amends) . 

[0068] Drawing 9 is the example which operates by this example, and shows the 
example of actuation of a window here. 

[0069] The window whose 90 is an object, and 91 are the size adjustable boxes for 
changing the magnitude of a window 90, a configuration, etc., and a user can do 
adjustable [ of the magnitude or the configuration of a window ] to arbitration by 
usually clicking and dragging with a mouse. Usually, when a size adjustable box is 
located at the lower right and carries out drag actuation of this in this way, the 
location of the upper left hand corner of a window is fixed, and the magnitude of a 
window 90 thru/or a configuration are determined as the rectangle which has the 
diagonal line formed by the coordinate of the upper left hand corner of the 
coordinate window of a final size adjustable box (the location on the configuration 
of the size adjustable box itself and a window etc. is not limited to this, but 
various methods are used). 

[0070] Now, as for a window, in the usual case, it is common that are considered and 
designed so that an inner display may be enlarged in order to make [ more ] the 
contents to display, and the size adjustable box of this window is also designed by 
not an exception but as small the configuration as possible. However, the one where 
a size adjustable box is larger is user-friendly for operating it with a mouse, 
having mentioned above and conflict arise, and as the large viewing area in a window 
is taken, actuation of a size adjustable box becomes more difficult. 
[0071] Such conflict is solvable with this example. . 

[0072] When the size of the window of drawing 9 is changed so that clearly from 
control of drawing 8 , A pointer 35 the size adjustable box 91 by that in which 
(step S72) and the reaction force from a mouse are generated by passing through 
passage or near (step 573) A user operates a click or a drag of a carbon button etc. 
in response to this reaction force feedback (feel) (it detects at step S76). if the 
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elapsed time (that is, — from passage of a size adjustable box or the near passage) 
from reaction force generating in the actuation time is in predetermined time, the 
coordinate at the time of button grabbing will be transposed to the coordinate 
information memorized at step S74 (step S77). 

[0073] therefore, it **** with what operated the object object even if that reaction 
had separated from the object object when a user' s reaction (button grabbing) was 
obtained by the predetermined within a time one according to reaction force 
generating according to this reaction force, and since the coordinate of an actuated 
valve position is amended like the time of operating an object object, the same 
effectiveness as the case where an object object is operated can be acquired. 
[0074] By this, a pointer is placed and dragged to inside, the same effectiveness as 
the time of clicking is acquired, actuation in which a user positions a pointer in a 
size adjustable box correctly becomes unnecessary in the case of a size adjustable 
box, and it becomes user-friendly user interface equipment for a user. 
[0075] That is, according to this example, even if a user does not arrange a pointer 
on an object completely, pass the top or near once and a user is made to produce 
reaction force, and when the need of concentrating cautions on the display of a 
screen conventionally is lost that what is necessary is just to carry out the 
depression of the mouse button and a user manipulates a graphical user interface by 
the response and feel of the reaction force, the burden of the user' s eye can be 
mitigated. 

[0076] In addition, in this example, rate decision of step S71 can combine 
processing of steps S74-S77, even when performing only location detection like the 
1st example instead of requirements required for an essential target about amendment 
of a coordinate location. 

[0077] The <4th example> In the above-mentioned examples 1-3, the mouse was made to 
generate frictional force to the detection ball of a mouse in a reaction force 
control section, damping force was applied to advance of a mouse, and the 
information from a host computer to a user is fed back. 

[0078] However, a means to feed back information to a user is not limited to an 
above-mentioned reaction force control section. In this example, an excitation means 
to vibrate a mouse mechanically is established and the configuration which feeds 
back information to a user is illustrated. 

[0079] Drawing 10 is the partial fracture side elevation of the mouse of this 
example, and explains only a difference with drawing 2 of the 1st example here. 
[0080] In drawing 10 , by passing a current, signs 201 are the direct-current motor 
in which a roll control is possible, and the eccentric weight which is made to carry 
out eccentricity of 202 intentionally to the revolving shaft of DC motor 201, and is 
attached, and can excite the whole mouse by carrying out the roll control of DC 
motor 201. 200 is an excitation control section possessing a DC motor and eccentric 
weight. 

[0081] The same result is obtained by replacing the reaction force control section 1 
and this excitation control section 200 of the already explained examples 1-3. 
[0082] Moreover, weight is vibrated with one voice coil other than a method using a 
DC motor and eccentric weight as an excitation means used by the excitation control 
section 200. 

[0083] 2) Strike the inside of a mouse mechanically. 

[0084] 3) Repeat frequently reaction force control of examples 1-3. 



8 of 10 



7/31/06 9:45 AM 



JP,09-026850,A [DETAILED DESCRIPTION] http://ww4.ipdl.ncipi.gojp/cgi-bin/tran_web_cgi_ejje 

[0085] 4) Since a mouse is used with a mouse pad (that with which the bottom of a 
mouse is covered) in many cases, it usually establishes the source of excitation in 
the mouse pad. 

[0086] Which configuration can be considered, and as long as it is the means which 
can feed back vibration to a user' s hand in short, what kind of thing may be 
adopted. 

[0087] Furthermore, in case it is used on the graphical user interface shown by 
drawing 3 by using these excitation means, excitation is carried out when one user 
drags objects, such as an icon and a window. 

[0088] 2) By establishing the means which carries out adjustable [ of the 
reinforcement of excitation ] to this excitation means, it becomes applicable [ 
changing the reinforcement of excitation etc. ] with the property of an object, for 
example, the file type and capacity which were related with the object. It cannot be 
overemphasized that the feedback by the reaction force in the 1st - the 3rd example 
is completely [ feedback by the excitation of this example ] replaceable. 
[0089] the example has described the configuration in the case of using a mouse as a 
pointing device since each ** of the <5th example) above. It cannot be 
overemphasized that the same configuration can be carried out also in pointing 
devices other than a mouse. 

[0090] Drawing 11 shows the computer of the laptop (note thru/or supplementary class 
note) mold which has spread in recent years, and has some which form the so-called 
trackball 301 which performs a coordinate input by operating a spherical ball by 
such computer 300. 

[0091] In drawing 11 , 300 is the computer of this example, 301 is a trackball, and 
it has a feedback means (un-illustrating) to feed back information to a user by 
mechanical works, such as a coordinate detection means, a control means, or an 
excitation means. The structure of a trackball may think that the detection ball of 
a mouse was operated by the direct hand, and can use for the feedback means for 
trackballs the almost same structure as the feedback means of the mouse mentioned 
above. 

[0092] The liquid crystal display with which a sign 302 displays the graphical user 
interface of drawing 3 , and 303 are keyboards which input a character code. 
[0093] If it is made to perform above-mentioned control using feedback means, such 
as the same excitation means as the excitation control section 3 stated in the same 
reaction force means or same example 4 as the reaction force control section 1 
stated to the trackball 301 in the 1st example, the effectiveness described until 
now and the same effectiveness will be acquired. 

[0094] When using an excitation means by this example, excitation of the keyboard 
near [ where excitation of the ball of a trackball itself is carried out, or a hand 
is touched by actuation of a computer ] the trackball may be carried out. 
[0095] In addition, if it is the pointing device which users, such as a tablet, and 
the AKYU point, a joy stick, operate by hand, the means which feeds back to a user' s 
body by the mechanical work as mentioned above can be established, and the same 
effectiveness as the above-mentioned example can be attained. 

[0096] Although the thing about the user interface by human being' s hand has been 
described until now, even if it uses parts other than a hand, for example, a guide 
peg etc., the same equipment configuration is possible. 
[0097] 
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' It 
■ 

[Effect of the Invention] As mentioned above, according to this invention, actuation 
of a user' s pointing device is received so that clearly. A feedback means to feed 
back information to a pointing device by mechanical works, such as reaction force, 
vibration, and an impact, at a user, In the operating environment using the 
graphical user interface which arranges objects, such as an icon, a window, and a 
pop up menu, and changes on a display In case the pointer operated by the pointing 
device passes and approaches the object for actuation, or its near, said feedback 
means is minded for the information about actuation of a pointing device. Since the 
configuration which established the control means which processes the information 
about actuation of the pointing device of the user according to this feedback is 
adopted while feeding back to a user While feeding back the information about 
actuation of a pointing device to a user through said feedback means Since the 
control means which processes the information about actuation of the pointing device 
of the user according to this feedback is established Since the user interface 
through means other than a visible display becomes possible, a user concentrates 
cautions on a display like before and it becomes unnecessary to manage the location 
of a pointer correctly, the user interface excellent in operability is realizable. 



[Translation done. ] 
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